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(54) Multi-functional on-vehicle camera system and image display method for the same 



(57) A multi-functional on-vehicle camera system 
comprises a plurality of cameras (11-18) mounted on a 
vehicle and monitors (19-21) each displaying a camera 
image. An image processing section (47) is provided for 
processing images taken by the plurality of cameras. A 
system control means (42) is provided for selecting a 
camera image to be processed from a plurality of 
images obtained from the pluralityof cameras based on 
signals (37-41) representing vehicle conditions, and for 
designating a type of image processing performed in 
the image processing section. The image for assisting 
the safety driving is displayed at an appropriate timing 
according to driving conditions in such a manner that 
the driver can easily grasp the circumferential condi- 
tions from the displayed image. 
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Description 

BACKGROUND OF THE INVENTION 

[0001] The present invention relates to a multi-func- 
tional on-vehicle camera system which comprises a plu- 
rality of monitoring cameras mounted on a vehicle and a 
display unit for displaying a monitored image. Further- 
more, the present invention relates to an image display 
method for this multi-functional on-vehicle camera sys- 
tem. More specifically, the present invention provides a 
monitor image display letting a driver easily know cir- 
cumferential conditions surrounding the vehicle. Moreo- 
ver, the present invention displays such images at an 
appropriate timing according to driving conditions. 
[0002] For the safety driving of automotive vehicles, 
a small camera is mounted on a vehicle and directed to 
a specific direction to take an image not directly viewed 
by a driver's eye. The taken image is displayed on a 
monitor in a passenger compartment of the vehicle. Fur- 
thermore, it is possible to take an image of white lines 
on a road and guide or control the vehicle based on the 
mutual position between the vehicle and the detected 
white lines. 

[0003] Moreover, the published unexamined Japa- 
nese patent application No. 7-215130, as shown in Fig. 
16, discloses a vehicle monitoring system comprising a 
plurality of cameras 1, 2, 3 and 4 mounted on a vehicle, 
a turning direction detecting section 5 for detecting a 
turning direction of the vehicle based on a winker signal, 
a steering direction detecting section 6 for detecting a 
steering direction of a steering wheel, an instruction 
input section 7 for allowing a driver to enter driver's 
instructions, a switching control section 8 for selecting 
an appropriate one of images taken by the plurality of 
cameras 1 , 2, 3 and 4 based on information obtained 
from the associated sections 5, 6 and 7, and an image 
display section (i.e., monitor) 9 for displaying the Image 
selected by the switching control section 8. 
[0004] However, in ordinary driving conditions of an 
automotive vehicle, a driver often applies a brake or 
steers a steering wheel to avoid a possible accident 
based on a quick judgement on an image displayed on 
the monitor. Accordingly, it is necessary to provide an 
easy-to-grasp monitor display so as to prevent the driver 
from being puzzled or making an erroneous judgement 
In this respect, the above-described conventional sys- 
tem should be more improved. 

[0005] For example, the images taken by cameras 
mounted on a front side of a vehicle may be selectively 
displayed in accordance with a turning of the vehicle to 
the right or to the left. In such a case, it is important for 
the driver to promptly understand the displayed image 
without taking a time to judge whether this image is a 
right side view seen from the vehicle or a left side view 
seen from the vehicle. Such a prompt judgement is 
essentially important for a skillful driving operation. 
[0006] For example, it may be desirable to display a 



word indicating the direction, e.g., "right" or "left", 
together with the displayed image. However, this forces 
the driver to carry out two thinking steps; i.e., one step 
of recognizing characters and another step of under- 

s standing the direction of the displayed image. A rela- 
tively long time Is required for these steps. It may be 
unable to avoid an accident due to delay in grasping 
information obtained from the displayed image. Further- 
more, the driver may not be able to quickly discriminate 

10 "right" from "left" or vice versa. 

[0007] Furthermore, when the vehicle moves back- 
ward to a vacant space in a parking lot, the driver may 
rely on an image taken by a camera mounted on a rear 
side of the vehicle. However, the intended vacant space 

15 may not be clearly displayed on the monitor due to pres- 
ence of another vehicles already in this parking lot In 
such a case, the driver cannot rely on the displayed 
image and will hesitate in steering the vehicle to a target 
place while avoiding obstacles. 

20 

SUMMARY OF THE INVENTION 

[0008] To solve the above-described conventional 
problems, the present invention has an object to provide 
25 a multi-functional ori-vehicle camera system capable of 
displaying an image for assisting the safety driving at an 
appropriate timing according to driving conditions in 
such a manner that the driver can easily grasp the dis- 
played image. 

30 [0009] Furthermore, the present invention has an 
object to provide an image display method for such a 
multi-functional on-vehicle camera system. 
[0010] In order to accomplish the above and other 
related objects, the present invention provides a multi- 

35 functional on-vehicle camera system comprising a plu- 
rality of cameras mounted on a vehicle and a display 
unit displaying at least one camera image, wherein 
Image processing means is provided for processing 
images taken by the plurality of cameras, and system 

40 control means is provided for selecting at least one 
camera image to be processed in the image processing 
means from the images taken by the plurality of cam- 
eras based on vehicle conditions, and for controlling a 
type of image processing performed in the image 

45 processing means. 

[0011] According to a preferred embodiment of the 
present invention, the multi-functional on-vehicle cam- 
era system further comprises graphic superimposing 
means for superimposing a graphic pattern on the 

so image processed by the image processing means. The 
system control means controls both a type and a posi- 
tion of the graphic pattern superimposed by the graphic 
superimposing means. 

[0012] According to the preferred embodiment of 
55 the present invention, it is preferable that the plurality of 
cameras include at least two cameras selected from the 
group consisting of a rear camera taking a rear view 
seen from the vehicle, right and left pillar cameras tak- 
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ing right and left rear side views seen from the vehicle, 
right and left door mirror cameras taking right and left 
door mirror views seen from the vehicle via right and left 
door mirrors, right and left lateral cameras taking front 
side views seen from the vehicle, and a front camera 
taking a front view seen from the vehicle. 
[0013] According to the preferred embodiment of 
the present invention, it is preferable that the display unit 
includes a right monitor displaying an image taken by a 
right door mirror camera, and a left monitor displaying 
an image taken by a left door mirror camera. 
[0014] According to the preferred embodiment of 
the present invention, it is preferable that the image 
processing means produces a plan view image based 
on images of a plurality of cameras so that the plan view 
image includes an image of the vehicle seen from 
above. 

[0015] According to the preferred embodiment of 
the present invention, it is preferable that the image 
processing means produces a panorama image based 
on images of a plurality of cameras. 
[0016] According to the preferred embodiment of 
the present invention, it is preferable that the image 
processing means produces a joint image by combining 
images of a plurality of cameras. 
[0017] According to the preferred embodiment of 
the present invention, it is preferable that the image 
processing means produces a surrounding image rep- 
resenting a surrounding view seen from the vehicle by 
successively connecting images of all of the plurality of 
cameras mounted on the vehicle. 
[0018] According to the preferred embodiment of 
the present invention, it is preferable that the image 
processing means produces an image formed by using 
part of image data obtained from the plurality of cam- 
eras. 

[0019] According to the preferred embodiment of 
the present invention, it Is preferable that the image 
processing means produces an image including a mov- 
ing object approaching or leaving with respect to the 
vehicle so that the moving object is emphasizing by a 
different color. "> 
[0020] According to the preferred embodiment of 
the present invention, it is preferable that the system 
control means designates the camera image to be used 
in the image processing and designates the type of 
image processing performed in the image processing 
means based on at least one signal representing vehi- 
cle conditions selected from the group consisting of a 
steering angle signal, a vehicle speed signal, gear posi- 
tion information and winker information. 
[0021] According to the preferred embodiment of 
the present invention, it is preferable that the graphic 
superimposing means superimposes a graphic pattern 
of a moving object approaching or leaving with respect 
to the vehicle. 

[0022] Furthermore, the present invention provides 
an image display method for a multi-functional on-vehi- 



cle camera system comprising a plurality of cameras 
mounted on a vehicle and a display unit displaying at 
least one camera image, the image display method 
comprising a step of processing at least one camera 
5 image selected from the a plurality of images obtained 
from the plurality of cameras based on vehicle condi- 
tions, and a step of displaying a processed camera 
image on the display unit. 

[0023] According to the preferred embodiment of 
10 the present invention, it is preferable that the plurality of 
cameras Include at least two cameras selected from the 
group consisting of the rear camera, the right and left 
pillar cameras, the right and left door mirror cameras, 
the right and left lateral cameras, and the front camera. 
15 And, the images of at least two cameras are processed 
in the image processing means and displayed on the 
display unit 

[0024] According to the preferred embodiment of 
the present invention, it is preferable that a plan view 
20 image is produced and displayed based on at least 
three images of the rear camera and the right and left 
pillar cameras when the vehicle moves backward for 
parking. 

[0025] According to the preferred embodiment of 

25 the present invention, it is preferable that a plan view 
image is produced and displayed based on at least 
three images of the rear camera, the left pillar camera 
and the left door mirror camera when the vehicle moves 
backward for parking along a left side of a road. 

30 [0026] According to the preferred embodiment of 
the present invention, it is preferable that a plan view 
image is produced and displayed based on at least 
three images of the rear camera, the right pillar camera 
and the right door mirror camera when the vehicle 

35 moves backward for parking along a right side of a road. 
[0027] According to the preferred embodiment of 
the present invention, it is preferable that a panorama 
image is produced and displayed by combining at least 
two images of the left door camera and the left pillar 

40 camera when the vehicle turns left, thereby eliminating 
an accident occurring at an inside of the vehicle during 
a left turning or cornering operation. 
[0028] According to the preferred embodiment of 
the present invention, it is preferable that a panorama 

45 image is produced and displayed by combining at least 
two images of the right door camera and the right pillar 
camera when the vehicle turns right, thereby eliminating 
an accident occurring at an inside of the vehicle during 
a right turning or cornering operation. 

50 [0029] According to the preferred embodiment of 
the present invention, it is preferable that a panorama 
image is produced and displayed by combining at least 
two images of the left pillar camera and the rear camera 
when the vehicle performs lane changing to a left lane. 

55 [0030] According to the preferred embodiment of 
the present invention, it is preferable that a panorama 
image is produced and displayed by combining at least 
two images of the right pillar camera and the rear cam- 
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era when the vehicle performs lane changing to a right 
lane. 

[0031] According to the preferred embodiment of 
the present Invention, it is preferable that the display unit 
includes a right monitor displaying an image taken by s 
the right door mirror camera and a left monitor display- 
ing an image taken by the left door mirror camera. 
[0032] According to the preferred embodiment of 
the present invention, it is preferable that each of the 
right and left monitors displays a limited image seen 10 
within a view angle of a corresponding door mirror when 
the vehicle is traveling. 

[0033] According to the preferred embodiment of 
the present invention, it is preferable that a panorama 
image is produced and displayed by combining three 15 
images of the right and left pillar cameras and the rear 
camera when the vehicle is traveling ordinarily. 
[0034] According to the preferred embodiment of 
the present invention, it is preferable that a joint image is 
produced and displayed by combining two images of the 20 
right and left lateral cameras when the vehicle is 
traveling slowly or stopped for confirmation of forward or 
side obstacles. 

[0035] According to the preferred embodiment of 
the present invention, it is preferable that the image of 25 
the right lateral camera is enlarged when the vehicle 
turns right, and the image of the left lateral camera is 
enlarged when the vehicle turns left. 
[0036] According to the preferred embodiment of 
the present invention, it is preferable that the image dis- 30 
played on the display unit includes a moving object 
approaching or leaving with respect to the vehicle, and 
the object is emphasizing by a different color. 
[0037] According to the preferred embodiment of 
the present invention, it is preferable that a graphic pat- 35 
tern is superimposed on the displayed image so that the 
graphic pattern represents a moving object approaching 
or leaving with respect to the vehicle. 
[0038] According to the preferred embodiment of 
the present invention, it is preferable that the image 40 
taken by the door mirror camera is continuously dis- 
played when any passenger is in the vehicle. 
[0039] With the characteristic features of the 
present invention, it becomes possible to display an 
image for assisting the safety driving at an appropriate 45 
timing according to driving conditions in such a manner 
that the driver can easily grasp the displayed image. 

BRIEF DESCRIPTION OF THE DRAWINGS 

50 

[0040] The above and other objects, features and 
advantages of the present invention will become more 
apparent from the following detailed description which is 
to be read in conjunction with the accompanying draw- 
ings, in which: 55 

Fig. 1 is a block diagram showing a multi-functional 
on-vehicle camera system in accordance with a 



preferred embodiment of the present invention; 
Fig. 2 is a flowchart showing an operation of the 
multi-functional on-vehicle camera system in 
accordance with the preferred embodiment of the 
present invention; 

Fig. 3 is a plan view showing a plurality of cameras 
mounted on the system vehicle in accordance with 
the preferred embodiment of the present invention; 
Rg. 4 is a front view showing a plurality of monitors 
installed in the system vehicle in accordance with 
the preferred embodiment of the present invention; 
Rg. 5 is a plan view showing layout of a plurality of 
cameras used for forming a plan view image in 
accordance with the preferred embodiment of the 
present invention; 

Rg. 6 is a view showing the procedure for forming 
the plan view image in accordance with the pre- 
ferred embodiment of the present invention; 
Rg. 7 is a plan view showing restriction of a view 
angle in each door mirror camera in accordance 
with the preferred embodiment of the present inven- 
tion; 

Rg. 8 is a view showing a panorama image formed 
in accordance with the preferred embodiment of the 
present invention; 

Rg. 9 is a view showing a joint image formed in 
accordance with the preferred embodiment of the 
present invention; 

Rg. 1 0 is a block diagram showing the arrangement 
of an image processing section performing the 
image processing for discriminately displaying an 
approaching object and a leaving object based on 
the image of a camera butputting R, G and B sig- 
nals in accordance with the preferred embodiment 
of the present invention; 

Rg. 1 1 is a block diagram showing the arrangement 
of an image processing section performing the 
image processing for discriminately displaying an 
approaching object and a leaving object based on 
the image of a camera outputting an NTSC signal in 
accordance with the preferred embodiment of the 
present invention; 

Rg. 12 is a block diagram showing a system 
arrangement for displaying a plan view image in 
accordance with the preferred embodiment of the 
present invention; 

Rg. 13 is a block diagram showing a system 
arrangement for displaying a joint image In accord- 
ance with the preferred embodiment of the present 
invention; 

Rg. 14 is a block diagram showing a system 
arrangement for displaying a panorama image in 
accordance with the preferred embodiment of the 
present invention; 

Rg. 15 is a block diagram showing a system 
arrangement for displaying a surrounding image in 
accordance with the preferred embodiment of the 
present invention; and 
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Fig. 16 is a block diagram showing a system 
arrangement of a conventional on- vehicle camera 
system. 

DESCRIPTION OF THE PREFERRED EMBODIMENT 

[0041] Hereinafter, a preferred embodiment of the 
present invention will be explained with reference to the 
attached drawings. Identical parts are denoted by the 
same reference numerals throughout the views. 
[0042] As shown In Fig. 3, the multi-functional on- 
vehicle camera system in accordance with a preferred 
embodiment of the present invention comprises a total 
of eight cameras mounted circumferentially on a vehi- 
cle. A front camera 1 1 , located at a front center of the 
vehicle, takes an image of a front view seen from the 
vehicle. A left lateral camera 12, located at a front left 
corner of the vehicle, takes a front left side view seen 
from the vehicle. A right lateral camera 1 3, located at a 
front right corner of the vehicle, takes a front right side 
view seen from the vehicle. A left door mirror camera 
1 4, located on or in the vicinity of a left door mirror, takes 
a left door mirror view seen from the vehicle via a left 
door mirror. A right door mirror camera 15, located on or 
in the vicinity of a right door mirror, takes a right door 
mirror view seen from the vehicle via a right door mirror. 
A left pillar camera 16, located on or in the vicinity of a 
left rear pillar, takes a left rear side view seen from the 
vehicle. A right pillar camera 17, located on or in the 
vicinity of a right rear pillar, takes a right rear side view 
seen from the vehicle. A rear camera 18, located at a 
rear center of the vehicle, takes a rear view seen from 
the vehicle. 

[0043] Fig. 4 shows a total of three monitors 
installed in a passenger compartment of the vehicle. A 
main monitor 19 is embedded in a dash board. A left 
view monitor 20 and a right view monitor 21 are installed 
near the upper end of a front windshield glass. The main 
monitor 1 9 has a screen size larger than those of the left 
and right view monitors 20 and 21 . 
[0044] Fig. 1 is a block diagram showing a sche- 
matic arrangement of a vehicle incorporating the multi- 
functional on-vehicle camera system of the present 
invention. Each of the above-described cameras 11 to 
18 mounted on the vehicle is constituted by a solid 
image pickup element, such as CCD or CMOS image 
pickup element An image data transmitting section 31, 
associated with a corresponding camera, converts a 
video signal of this camera into image data and trans- 
mits the converted image data via a vide bus 35. An 
image processing section 47 receives the image data 
transmitted via the video bus 35 to analyze the image 
data and perform editing and composing processing. A 
memory 48 is used as a work area for the image 
processing. A plurality of D/A converting sections 50-52 
convert the processed image data into an analog signal. 
A character generator 46 produces a graphic pattern 
displayed in accordance with a code. A character gener- 



ator control section 45 controls the graphic pattern pro- 
duced from the character generator 46. A mixer 49 
mixes the graphic pattern produced from the character 
generator 46 with the image produced from the D/A 

5 converter 50. The main monitor 1 9, the left view monitor 
20, and the right view monitor 21 display the image. A 
camera control section 43 controls the direction/angle of 
each of the cameras 1 1 -1 8 as well as image pickup con- 
ditions. An alarm generating section 44 raises the 

10 alarm. A system control section 42 controls various sec- 
tions based on driver's instructions entered from an 
operating section 36, a steering angle signal 37, a vehi- 
cle speed signal 38, gear position information 39, a 
winker signal 40 and a supersonic distance sensor 41 . 

15 [0045] Furthermore, each image data transmitting 
section 31 comprises an A/D converting section 32 con- 
verting the video signal of each camera into a digital sig- 
nal, a frame memory 34 storing image data, and a 
frame memory control section 33 controlling storage 

20 and transmission of the image data. 

[0046] The cameras 11-18 produce various video 
signals in conformity with NTSC standards, PAL stand- 
ards etc. Or, the camera output signals may meet the 
VGA standards. The video signals for each standard 

25 include R,G and B outputs, Y, U and V outputs, a com- 
posite output, as well as digital outputs. The digital out- 
puts require no A/D and D/A converters. 
[0047] The system control section 42 inputs various 
sensing signals; e.g., the steering angle signal 37 repre- 

30 senting a steering angle of a steering wheel, the vehicle 
speed signal 38 representing a rotational speed of a 
wheel, the gear position information 39 representing a 
gear position in a power train mechanism of the vehicle, 
the winker signal 40 responsive to a right or left turning 

35 lever, and the supersonic distance sensor 41 located at 
a front or rear side of the vehicle to detect a distance 
from this vehicle to a neighboring vehicle or obstacle. 
[0048] The system control section 42 controls each 
image data transmitting section 31 based on the 

40 obtained information as well as the driver's instructions 
entered from the operating section 36. The system con- 
trol section 42 sends selected image data obtained from 
the cameras 11-18 to the image processing section 47. 
Furthermore, the system control section 42 controls the 

45 way of processing images in the image processing sec- 
tion 47. To this end, the system control section 42 con- 
trols the camera control section 43 to adjust the 
direction/angle of each camera, thereby obtaining nec- 
essary image data. 

so [0049] The image processing section 47 analyzes 
the received image data, edits the image data, and 
composes the images based on instructions sent from 
the system control section 42. Furthermore, the image 
processing section 47 sends required analysis result to 

55 the system control section 42. The system control sec- 
tion 42 activates the alarm generating section 44 and 
controls the character generator control section 45 if 
necessary based on the obtained analysis result. 
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[0050] The character generator control section 45, 
when it receives an instruction from the system control 
section 42, causes the character generator 46 to output 
a graphic pattern to be superimposed on the image 
processed by the image processing section 47. 5 
[0051] To assist the safety driving, the monitors 19 
to 21 display the images at an appropriate timing 
according to driving conditions in such a manner that 
the driver can easily grasp the displayed image. 
[0052] Hereinafter, the image processing per- w 
formed chiefly in the Image processing section 47 will 
be explained in greater detail. 

Plan View Image 

[0053] The plan view image, i.e., an image of the 
vehicle and surroundings seen from above, is obtained 
by analyzing and composing the plurality of images 
obtained from the cameras mounted on the vehicle. For 
example, when the vehicle moves backward for parking, 20 
the main monitor 19 displays the plan view image of a 
parking lot seen from above. Thus, the driver can clearly 
recognize a vacant space in the parking lot based on 
the image displayed on the main monitor 1 9. A non-pub- 
lished Japanese patent application No. 11-109946 dis- 25 
closes details about the method for creating the plan 
view image. 

[0054] Rg. 6 shows the procedure of forming the 
plan view image by using the images obtained from the 
door mirror cameras 14 and 15, the pillar cameras 16 30 
and 1 7, and the rear camera 1 8 shown in Fig. 5. Accord- 
ing to Rg. 6, the left side and the central portion of a 
composite image is formed based on the images 
obtained from the left door mirror camera 14, the left pil- 
lar camera 1 6, and the rear camera 18. 35 
[0055] Rrst, the images obtained from the left door 
mirror camera 1 4, the left pillar camera 1 6, and the rear 
camera 18 are converted into projective images pro- 
jected on the road. To this end, a road area is deter- 
mined so as to correspond to the field of view of each 40 
camera shown in Fig. 5. The image taken by each cam- 
era is converted into the projected-on-road image 
through coordinate conversion. Each projected-on-road 
image thus obtained is weighted in accordance with an 
a map which indicates an area necessary for forming 4$ 
the plan view image. The weighted images are then 
combined to form a composite image. 
[0056] The images of the right door mirror camera 
1 5 and the right pillar camera 1 7 are processed in the 
same manner. Thus, all of the weighted images being so 
weighted by a maps are combined to form a composite 
image. A graphic image representing an automotive 
vehicle is disposed on a vehicle position in the obtained 
composite image. The memory 48 stores the graphic 
image representing an automotive vehicle. 55 
[0057] In this manner, the plan view image is cre- 
ated. The obtained plan view image includes the system 
vehicle and surroundings. 
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[0058] The above-described plan view image 
emphasizes the rear side of the system vehicle based 
on the images of the door mirror cameras 1 4 and 1 5, the 
pillar cameras 16 and 17 and the rear camera 18. It is 
however possible to obtain a plan view image empha- 
sizing the left side and the left rear side of the system 
vehicle based on the images of the left lateral camera 
12, the left door mirror camera 14, the left pillar camera 
16, and the rear camera 18. Similarly, it is possible to 
obtain a plan view image emphasizing the right side and 
the right rear side of the system vehicle from the images 
of the right lateral camera 1 3, the right door mirror cam- 
era 15, the right pillar camera 17, and the rear camera 
1 8. These two plan image views serve as useful image 
information for assisting the driver when parking along 
left and right sides of a road. 

[0059] The number of cameras used for forming the 
plan view image may be two or less. 

Panorama Image 

[0060] The panorama image is a super wide image 
obtained by combining a plurality of images taken by the 
cameras mounted on the vehicle. More specifically, the 
panorama image has a view angle of approximately 
1 60-1 70° whereas an ordinary wide image has a view 
angle of approximately 120°. 

[0061] Fig. 8 shows a method of forming the pano- 
rama image. For example, an image 91 of the right door 
mirror camera 15 and an image 92 of the right pillar 
camera 17 are analyzed to detect image elements com- 
monly involved in the analyzed images 91 and 92. Then, 
a panorama image is composed by overlapping the 
common elements involved in both images 91 and 92. 
When the resultant panorama image is displayed on the 
main monitor 19, the driver can grasp a wide range of 
circumferential conditions around the vehicle at a 
glance. The panorama Image Is advantageous com- 
pared with the normal images having more narrower 
view angles in that the driver can immediately under- 
stand the relationship between the actual position and 
the displayed image. Hence, in a urgent situation where 
the driver's prompt operation based on the displayed 
image is required, the panorama image makes it possi- 
ble for the driver to perform the vehicle driving operation 
properly without any hesitation. 

[0062] In the formation of the panorama image, it is 
necessary to adjust the direction/angle of each camera 
so that two cameras are in a partly overlapped relation- 
ship in their fields of view while providing a sufficiently 
wide view angle when their fields of view are integrated. 
Accordingly, when the panorama image is displayed, 
the system control section 42 instructs the camera con- 
trol section 43 to perform the direction/angle adjustment 
for respective cameras 11-18. 

[0063] Furthermore, it is possible to combine three 
or more camera images to produce a panorama image. 
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Surrounding Image 

[0064] The surrounding image is formed by succes- 
sively connecting all the images taken by the cameras 
11-18 based on a driver's instruction of the operating 5 
section 36. Thus, the surrounding image is an image 
continuously displaying surroundings or circumstances 
around the vehicle. The operating section 36 is a joy- 
stick type which allows the driver to instruct a preferable 
direction. When the driver instructs any preferable dlrec- w 
tion, the system control section 42 causes the image 
data transmitting sections 31 of appropriate neighboring 
cameras, e.g., camera 1 1 and camera 12, located in the 
driver's instructing direction to output images. The 
image processing section 47 connects the. output is 
images by overlapping the same image elements as 
performed in the formation of the panorama image. 
When the driver's instructing direction is changed con- 
tinuously in the clockwise direction, the images of the 
cameras 1 2 and 14 are connected and then the images 20 
of the cameras 14 and 16 are connected. In this man- 
ner, neighboring images are successively connected 
one by one. The image processing is performed in such 
a manner that the displayed images are continuously 
scrolled in one direction on the monitor screen. Accord- 4 25 
ingly, when the driver's instruction for the surrounding 
display is entered from the operating section 36, the 
main monitor 19 displays the surrounding image suc- 
cessively along an entire circumference of the vehicle. 
[0065] Like the above-described formation of the 30 
panorama image, the camera control section 43 adjusts 
the direction/angle of respective cameras 11-18 based 
on the instruction sent from the system control section 
42 so as to optimize the resultant surrounding image. 

35 

Joint Image 

[0066] The joint image is a combination of two dis- 
continuous images with a clear boundary thereof. For 
example, the driver may intend to turn the vehicle right 40 
in a slow speed condition. In such a case, the main 
monitor 19 displays a joint image consisting of the 
image taken by the left lateral camera 12 and the image 
taken by the right lateral camera 1 3 as shown in Fig. 9A. 
[0067] In this manner, by displaying two images as 45 
a joint image, the camera direction of each displayed 
image can be grasped perceptually. Furthermore, the 
conditions seen from the vehicle in different directions 
can be recognized at a glance. This makes it possible to 
quickly obtain the circumferential information of the right so 
and left sides of the road. Such information is usually 
unobtainable unless the driver swings his/her head in 
both right and left directions. 

[0068] As shown in Fig. 9B, when the vehicle turns 
left, the main monitor 19 enlarges the image taken by ss 
the left lateral camera 12 compared with the image 
taken by the right lateral camera 13. On the contrary, as 
shown in Fig. 9C, when the vehicle turns right, the main 



monitor 19 enlarges the image taken by the right lateral 
camera 13 compared with the image taken by the left 
lateral camera 12. Thus, the driver can perceive the 
camera direction of each displayed image intuitionally in 
connection with the advancing direction of the vehicle. 
In this case, the image corresponding to the turning 
direction of the vehicle is enlarged because the driver 
should pay a greater caution to the turning direction of 
the vehicle. Thus, the driver can unconsciously find the 
image to be paid attention. 

View Field Limited Image 

[0069] The view field limited image is a limited 
image taken by each of the door mirror camera 1 4 and 
1 5, corresponding an image seen within a view angle of 
a door mirror when the vehicle is traveling, as shown in 
Fig. 7. To this end, the image processing section 47 
abandons the image data outside a door mirror effective 
area from the image data of each of the door mirror 
cameras 14 and 15. Furthermore, the image processing 
section 47 enlarges the image data of the door mirror 
effective area to a full range of the screen. After being 
processed in this manner, the limited image taken by 
the left door mirror camera 14 is displayed on the left 
view monitor 20. Similarly, the limited image taken by 
the right door mirror camera 15 is displayed on the right 
view monitor 21 . 

[0070] The reason why the images of the door mir- 
ror cameras 14 and 15 are converted into the view field 
limited images is to effectively utilize the drive feeling 
acquired through driver's experiences. The driver can 
look the displayed images of the left view monitor 20 
and the right view monitor 21 without any unusual feel- 
ing. In other words, the information obtained from the 
door mirror cameras 14 and 15 is substantially identical 
with the information obtained from the door mirrors. In 
actual driving operation, the image information outside 
the door mirror effective area is not so useful in the vehi- 
cle traveling condition. It is thus believed that clearly dis- 
playing the image of the door mirror effective area can 
provide a meaningful display. 

Addition of Image Information 

[0071] To raise driver's caution, or to aid driver's 
understanding the displayed image, a supplementary 
graphic pattern is added or part of the displayed image 
is modified. The following is practical examples of addi- 
tional image information. 

(a) Discriminating Approaching Object and Leaving 
Object 

[0072] It is, for example, desirable that a red frame 
is put on an automotive vehicle approaching to the sys- 
tem vehicle while a green frame is put on a vehicle or 
other object leaving from the system vehicle. 
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[0073] To this end, the image processing section 47 
detects a motion vector based on the frame-by-frame 
comparison on the image data taken by the front cam- 
era 1 1 . The image processing section 47 obtains coor- 
dinate values of an object approaching to or leaving 
from the system vehicle. The image processing section 

47 transmits the obtained coordinate values of the 
approaching or leaving object to the system control sec- 
tion 42. The system control section 42 instructs the 
character generator control section 45 to output a red or 
green frame and designates its position. The character 
generator control section 45 controls the character gen- 
erator 46 to outputs a designated graphic pattern to a 
designated position. The graphic pattern produced from 
the character generator 46 is sent to the mixer 49. The 
mixer 49 mixes the received graphic pattern with the 
image processed by the image processing section 47. 
The main monitor 19 displays the resultant image 
including an approaching object encircled by the red 
frame and a leaving object encircled by the green frame. 
[0074] It is however possible to emphasize the 
approaching object itself instead of putting the graphic 
pattern (i.e., red frame). For example, the approaching 
object itself can be emphasized by red and the leaving 
object can be emphasized by green. 

[0075] Fig. 1 0 is a functional block diagram showing 
the arrangement of the image processing section 47 to 
realize such display. The image processing section 47 
comprises a binary coding section 61 for converting an 
input image into a binary-coded image. A comparing 
section 62 compares the binary-coded image entered 
from the binary coding section 61 with a reference value 
stored in the memory 48. The reference value is a 
binary-coded image entered one field before. An aver- 
age calculating section 63 calculates an average shift 
amount of image segments based on the comparison 
result of the comparing section 62. A subtracting section 
64 calculates a difference between a shift amount of 
each image segment and the average shift amount A 
moving object discriminating section 65 discriminates 
each moving object An image extracting section 66 
extracts the image of the detected moving object. 
[0076] Hereinafter, the image processing is per- 
formed based on RGB video signals of the front camera 
1 1. Of the RGB video signals outputted from the front 
camera 1 1 , the video signal of G channel is entered into 
the binary coding section 61 . The binary coding section 
61 compares a signal level of each pixel with a predeter- 
mined threshold level, to produce a binary-coded 
image. The binary-coded image Is sen to the memory 

48 and the comparing section 62. The comparing sec- 
tion 62 reads the binary-coded image of the immedi- 
ately previous field from the memory 48. The comparing 
section 62 compares the readout image with the binary- 
coded image received from the binary coding section 
61, thereby detecting a shift amount of each pixel. The 
average calculating section 63 obtains an average of 
the shift amounts of each pixel detected by the compar- 



ing section 62. The average thus obtained is equivalent 
to a relative shift amount of a stationary object relative 
to the system vehicle. 

[0077] The subtracting section 64 subtracts the 

s average calculated by the average calculating section 
63 from the shift amount of each pixel detected by the 
comparing section 62, thereby obtaining a difference of 
each pixel with respect to the average shift amount. 
[0078] The moving object discriminating section 65 

w inputs two threshold levels, I.e., "approaching object 
judging threshold level" and "leaving object Judging 
threshold level", which are determined beforehand. The 
moving object discriminating section 65 compares the 
average amount of each pixel with these threshold lev- 

15 els, thereby discriminating pixels displaying any 
approaching object as well as pixels displaying any 
leaving object. The discrimination information is sent to 
the image extracting section 66. 
[0079] The image extracting section 66 extracts the 

20 image of the detected moving object, i.e., the approach- 
ing object and the leaving object, from the binary-coded 
image. The image of the extracted "approaching object" 
is outputted to the R channel, and the image of the 
extracted "leaving object 9 is outputted to the G channel. 

25 [0080] The output image of the image extracting 
section 66 is superimposed on the image taken by the 
front camera 1 1 . The composed image is displayed on 
the monitor. As a result, the displayed image of the 
approaching object is emphasized by red, while the dis- 

30 played image of the leaving object is emphasized by 
green. 

[0081] Fig. 1 1 is another block diagram showing the 
arrangement of the image processing section 47 for 
processing a NTSC composite vide signal of the front 

35 camera 1 1 . The image processing section 47 comprises 
a band elimination filter 67, such as a chroma cut filter, 
which cuts color signal components from the NTSC sig- 
nal and extracts brightness signal components only. 
The binary coding section 61 produces a binary-coded 

40 image from the brightness signal received from the 
band elimination filter 67. The comparing section 62, 
the average calculating section 63, the subtracting sec- 
tion 64, and the moving object discriminating section 65 
operate in the same manner as those disclosed in Fig. 

45 10. 

[0082] The image extracting section 66 receives the 
discrimination information from the moving object dis- 
criminating section 65, and extracts the images of the 
approaching object and the leaving object from the 
so binary-coded image sent from the binary coding section 
61. The extracted image of the approaching object is 
turned to red, and the image of the leaving object is 
turned to green. 

[0083] The output signal of the image extracting 
55 section 66 is superimposed on the image signal pro- 
duced from the camera 1 1 . The composite image is dis- 
played on the monitor. As a result, the displayed image 
of the approaching object is emphasized by red, white 
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the displayed image of the leaving object is emphasized 
by green. 

[0084] The image taken by the rear camera 1 8 pro- 
vided at the rear end of the vehicle can be processed in 
the same manner as in the processing of the image of 
the front camera 1 1 . This realizes a display of a vehicle 
approaching to the system vehicle from behind as well 
as a vehicle leaving from the system vehicle to far 
behind. 

(b) Display of Traveling Track on Plan View Image 

[0085] A traveling track of a system vehicle is super- 
imposed on the plan view image displayed when the 
system vehicle moves backward. In this case, the image 
processing section 47 analyzes the image used for 
forming the plan view image and obtains graphic pattern 
data of a traveling track of the system vehicle as a pro- 
jection on the road displayed on the plan view image. 
The image processing section 47 sends the obtained 
graphic pattern data of a traveling track to the system 
control section 42. The system control section 42 
instructs the character generator control section 45 to 
display dots representing the traveling track of the sys- 
tem vehicle and designates their positions. Thus, the 
image of dots representing the traveling track of the sys- 
tem vehicle is superimposed on the plan view image. 
[0086] Furthermore, the system control section 42 
can calculate a predictive traveling track of the system 
vehicle based on the traveling track data obtained from 
the image processing section 47 and the steering angle 
signal 37. Thus, it is possible to superimpose the image 
of the predictive traveling tack on the plan view image 
via the character generator control section 45. Based on - 
the predictive traveling track displayed on the monitor, 
the driver can easily make a judgement as to whether or 
not the system vehicle can smoothly and safely enter a 
target vacant space for parking. 

[0087] Furthermore, by comparing the positional 
relationship between the predictive track based on the 
steering angle signal 37 and the target vacant space on 
the plan view image, it is possible to calculate an esti- 
mated position and an estimated angle of the system 
vehicle at the entry of the target vacant place. Thus, it 
becomes possible to teach the driver how to operate the 
steering wheel through the monitor display, or by using 
a voice guidance through the alarm generating section 
44. In this case, it is preferable to guide the system vehi- 
cle to reach the center of the entry of the target vacant 
space at the right angle. 

[0088] The system control section 42 displays 
these composite images in the following manner. 
[0089] Fig. 2 is a flowchart showing the operation of 
the system control section 42. 

[0090] In step 1 , it is checked whether or not the 
gear position information 39 indicates that the present 
gear position is a "back" position. 
[0091] When the present gear position is the "back" 



position (i.e., Yes in step 1), the control flow proceeds to 
a step 2. 

[0092] In step 2, it is checked whether or not the 
driver selects a parking assist mode through the operat- 
5 ing section 36. 

[0093] When the driver selects the parking assist 
mode (i.e., Yes in step 2), the control flow proceeds to a 
step 3. 

[0094] In step 3, the main monitor 19 displays a 
w plan view image based on the Images taken by the left 
pillar camera 1 6, the right pillar camera 1 7 and the rear 
camera 1 8. 

[0095] When the driver does not select the parking 
assist mode (i.e., No in step 2), the control flow pro- 
fs ceeds to a step 4. 

[0096] In step 4, it is checked whether or not the 
driver selects a column parking assist mode through the 
operating section 36. In this case, the column parking is 
a parking mode for parking a vehicle along a left or right 
20 side of a road (or along a building etc.) where parked 
vehicles aligns vertically. 

[0097] When the driver selects a left column park- 
ing assist mode, the control flow proceeds to a step 5. 
[0098] In step 5, the main monitor 19 displays a 
25 plan view image based on the images taken by the left 
door camera 14, the left pillar camera 16, and the rear 
camera 1 8. 

[0099] When the driver selects a right column park- 
ing assist mode, the control flow proceeds to a step 6. 
30 [0100] In step 6, the main monitor 19 displays a 
plan view image based on the images taken by the right 
door camera 15, the right pillar camera 1 7, and the rear 
camera 18. 

[0101] Fig. 12 is a block diagram showing a system 

35 arrangement for displaying a plan view image. The 
image processing section 47 comprises an image 
selecting section 72 for selecting the images of desig- 
nated cameras, a plan view image forming section 78 
for producing a plan view image based on the selected 

40 images, and a view angle converting section 75 for con- 
verting the view angles of the door mirror cameras 14 
and 15. The system control section 42 comprises a 
positional information detecting section 71 for detecting 
vehicle conditions based on the gear position informa- 

45 tion, the winker information, and the steering angle 
information. When the gear is shifted to the "back" posi- 
tion, the positional information detecting section 71 of 
the system control section 42 instructs the plan view 
image forming section 78 of the image processing sec- 

so tion 47 to produce a plan view image. Furthermore, the 
positional information detecting section 71 instructs the 
image selecting section 72 to select the images taken 
by cameras 16, 17 and 18 in response to the driver's 
choice of the parking assist mode. Furthermore, when 

55 the column parking assist mode is chosen, the winker 
information and the steering angle information are 
referred to select the images taken by cameras 14, 16 
and 18 for the left column parking or select the images 
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taken by cameras 15, 17 and 18 for the right column 
parking. Furthermore, the positional information detect- 
ing section 71 instructs the view angle converting sec- 
tion 75 to enlarge the portion corresponding to the door 
mirror effective area in the images taken by the door 5 
mirror cameras 14 and 15. 

[0102] The enlarged image of the left door mirror 
camera 14, being processed in the view angle convert- 
ing section 75, is displayed on the left view monitor 20 
via a switching section 76. The enlarged image of the 
right door mirror camera 15, being processed in the 
view angle converting section 75, is displayed on the 
right view monitor 21 via a switching section 77. The 
plan view image produced from the plan view image 
forming section 78 is displayed on the main monitor 1 9. 
[01 03] Returning to the flowchart of Fig. 2, when the 
present gear position is not the "back" position (i.e., No 
in step 1 ), the control flow proceeds to a step 7. 
[0104] In step 7, based on information of the vehicle 
speed signal 38, it is checked whether or not the vehicle 
speed is equal to or less than 8 km/h. 
[0105] When the vehicle speed is equal to or less 
than 8 km/h (i.e., Yes in step 7). it is assumed that the 
driver in the system vehicle intends to carefully watch 
the front or the sides. Thus, the control flow proceeds to 
a step 1 5. 

[0106] In step 15, the main monitor 19 displays the 
joint image based on the images taken by the left lateral 
camera 12 and the right lateral camera 13. 
[0107] Fig. 1 3 is a block diagram showing a system 
arrangement for displaying a joint image. The image 
processing section 47 comprises a joint image forming 
section 73 for producing a joint image in addition to the 
image selecting section 72 and the view angle convert- 
ing section 75. When the vehicle speed is equal to or 
less than 8 km/h, the positional information detecting 
section 71 of the system control section 42 instructs the 
image selecting section 72 of the image processing sec- 
tion 47 to select the images taken by cameras 12 and 
13. Furthermore, the positional information detecting 
section 71 instructs the joint image forming section 73 
to produce a joint image consisting of the images taken 
by cameras 12 and 13. When the operating section 
inputs an instruction for displaying a joint image consist- 
ing of images having different display sizes, the posi- 
tional information detecting section 71 informs the joint 
image forming section 73 of an image to be enlargedly 
displayed based on the gear position information. The 
joint image forming section 73 changes the display ratio 
of the images to be jointly displayed based on this 
instruction, and produces a modified joint image. 
[0108] Furthermore, the positional information 
detecting section 71 of the system control section 42 
instructs the view angle converting section 75 to directly 
display the images taken by the door mirror cameras 1 4 
and 1 5 without changing their display sizes. 
[0109] The wide angle image taken by the left door 
mirror camera 14 is displayed on the left view monitor 



20 via the switching section 76. The wide angle image 
of the right door mirror camera 15 is displayed on the 
right view monitor 21 via the switching section 77. The 
joint image produced from the joint image forming sec- 
tion 73 is displayed on the main monitor 19. 
[01 10] Returning to the flowchart of Fig. 2, when the 
vehicle speed is larger than 8 km/h (i.e., No in step 7), 
the control flow proceeds to a step 8. 
[0111] In step 8, the view angles of the door mirror 
cameras 14 and 15 are narrowed so as to be equalized 
to the door mirror effective area. 
[01 12] Then, the control flow proceeds to a step 9. 
[0113] In step 9, it is checked whether or not the 
vehicle speed is equal to or larger than 50 km/h. 
[0114] When the vehicle speed information indi- 
cates the speed equal to or larger than 50 km/h (i.e., 
Yes in step 9), the control flow proceeds to a step 10. 
[0115] In step 10, based on the winker signal 40, it 
is checked whether or not the winker is turned on. 
[0116] When the winker is. turned on (i.e., Yes in 
step 1 0), it is assumed that the driver Intends to perform 
lane changing. The control flow, proceeds to step 1 1 . 
[0117] In step 11, it is checked whether the winker 
signal 40 indicates left or right. When the winker signal 
40 indicates left, the control flow proceeds to a step 12. 
[0118] In step 12, the main monitor 19 displays a 
panorama image based on the images taken by the left 
pillar camera 16 and the rear camera 18. When the 
winker signal 40 indicates right, the control flow pro- 
ceeds to a step 13. 

[0119] In step 13, the main monitor 19 displays a 
panorama image based on the images taken by the 
right pillar camera 1 7 and the rear camera 1 8. 
[0120] When the winker is turned off (i.e., No in step 
1 0), the control flow proceeds to a step 1 4. 
[0121] In step 14, it is assumed that the system 
vehicle is in an ordinary traveling condition where the 
system vehicle substantially goes straight at a speed 
larger than 8 km/h. The main monitor 19 displays a pan- 
orama image based on the images taken by the left pil- 
lar camera 16, the rear camera 18, and the right pillar 
camera 1 7. 

[0122] When the vehicle speed is less than 50 km/h 
(i.e., No in step 9), the control flow proceeds to a step 
16. 

[0123] In step 16, based on the winker signal 40, it 
is checked whether or not the winker is turned on. 
[0124] When the winker is turned on (i.e., Yes in 
step 1 6), it is assumed that the driver intends to perform 
a turning or cornering operation of the system vehicle. 
In this case, to avoid any accident occurring at an inside 
of the turning or cornering vehicle, the control flow pro- 
ceeds to a step 17. 

[0125] In step 17, it is checked whether the winker 
signal 40 indicates left or right. When the winker signal 
40 indicates left, the control flow proceeds to a step 18. 
[0126] In step 18, the main monitor 19 displays a 
panorama image based on the images taken by the left 
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door mirror camera 14 and the left pillar camera 16. 
When the winker signal 40 indicates right, the control 
flow proceeds to a step 1 9. 

[0127] in step 13, the main monitor 19 displays a 
panorama image based on the images taken by the 
right door mirror camera 15 and the right pillar camera 
17. 

[0128] When the winker is turned off (i.e., Yes in 
step 16), the control flow proceeds to the step 14 in 
which the main monitor 1 9 displays a panorama image 
based on the images taken by the left pillar camera 1 6, 
the rear camera 18, and the right pillar camera 17. 
[0129] Fig. 14 is a block diagram showing a system 
arrangement for displaying a panorama image. The 
image processing section 47 comprises a panorama 
image forming section 74 for producing a panorama 
image in addition to the image selecting section 72 and 
the view angle converting section 75. When the vehicle 
speed information indicates the speed equal to or larger 
than 50 km/h and the winker information or the steering 
angle information indicates the left lane changing, the 
positional information detecting section 71 of the sys- 
tem control section 42 instructs the image selecting 
section 72 of the image processing section 47 to select 
the images taken by cameras 16 and 18. Similarly, 
when the winker information or the steering angle infor- 
mation indicates the right lane changing, the positional 
information detecting section 71 instructs the image 
selecting section 72 to select the images taken by cam- 
eras 17 and 18. 

[0130] Furthermore, when the vehicle speed infor- 
mation indicates the speed in the range from 8 km/h to 
50 km/h and the winker information or the steering 
angle information indicates the left turning or cornering 
operation, the positional information detecting section 
71 instructs the image selecting section 72 to select the 
images taken by cameras 14 and 16. Similarly, when 
the winker information or the steering angle Information 
indicates the right turning or cornering operation, the 
positional information detecting section 71 instructs the 
image selecting section 72 to select the images taken 
by cameras 15 and 17. 

[0131] Moreover, when the vehicle speed informa- 
tion indicates the speed equal to or larger than 8 km/h 
and the winker information or the steering angle infor- 
mation indicates that the system vehicle is traveling 
straight, the positional information detecting section 71 
instructs the image selecting section 72 to select the 
images taken by cameras 16, 17 and 18. 
[0132] Furthermore, the positional information 
detecting section 71 instructs the panorama image 
forming section 74 to display a panorama image based 
on the images selected by the image selecting section 
72. 

[0133] Furthermore, the positional information 
detecting section 71 instructs the view angle converting 
section 75 of the image processing section 47 to 
enlarge the portion corresponding to the door mirror 



effective area in the images taken by the door mirror 
cameras 14 and 15. 

[0134] The enlarged image of the left door mirror 
camera 14, being processed in the view angle convert- 

5 ing section 75, is displayed on the left view monitor 20 
via the switching section 76. The enlarged image of the 
right door mirror camera 15, being processed in the 
view angle converting section 75, is displayed on the 
right view monitor 21 via the switching section 77. The 

w panorama image produced from the panorama image 
forming section 74 is displayed on the main monitor 19. 
[01 35] In this manner, this system makes it possible 
to adequately switch the display pattern of images in 
accordance with the steering angle signal 37, the vehi- 

15 cle speed signal 38, the gear position information 39, 
the winker signal 40 etc., thereby realizing the useful 
display of images to assist the safety driving. 
[0136] Fig. 15 is a block diagram showing a system 
arrangement for displaying a surrounding image based 

20 on a driver's instruction. The positional information 
detecting section 71 of the system control section 42 
instructs the image selecting section 72 of the image 
processing section 47 to select the image of each cam- 
era designated by the driver via a jog controller 36. Fur- 

25 thermore, the positional information detecting section 
71 instructs the joint image forming section 73 to com- 
bine the images selected by the image selecting section 
72. When the driver changes the direction of the jog 
controller 36, the image selecting section 72 changes 

30 the image to be displayed in response to the directional 
change of the jog controller 36. The joint image forming 
section 73 produces a composite image. The resultant 
composite image is displayed on the left view monitor 
20, the right view monitor 21 , and the main monitor 19. 

35 [0137] When the system vehicle is stopped, a pas- 
senger may get off the vehicle incautiously and may be 
involved in an accident when other vehicle approaches 
from behind. To prevent such an accident, presence of 
any passenger in the system vehicle is checked by a 

40 sensor or a micro camera installed in a passenger com- 
partment. When any passenger stays in the system 
vehicle even after the key of this vehicle is turned off, the 
left view monitor 20 and the right view monitor 21 con- 
tinuously display the images taken by the door mirror 

45 cameras 14 and 15, respectively. 

[0138] The above-described embodiment is based 
on the eight cameras mounted on the vehicle. However, 
it is possible to increase or decrease the total number of 
the cameras. 

so [0139] In the system of the present invention, the 
relationship between the vehicle conditions and the 
images to be displayed is not limited to the one dis- 
closed in Fig. 2. Therefore, it is possible to variously 
change the relationship between the vehicle conditions 

55 and the images to be displayed. 

[01 40] As apparent from the foregoing explanation, 
according to the multi-functional on-vehicle camera sys- 
tem and the related image display method, it becomes 
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possible to display the images for assisting the safety 
driving at an appropriate timing according to driving 
conditions. Furthermore, the present invention makes it 
possible to display the images in such a manner that the 
driver can easily grasp the circumferential conditions 5 
from the displayed Image. 

[0141] Displaying the images in accordance with 
the present invention makes it possible for the driver to 
pay caution to circumferential conditions without any 
burden. This helps the driver avoiding any accident or 10 
trouble. 

[0142] This invention may be embodied in several 
forms without departing from the spirit of essential char- 
acteristics thereof. The present embodiment as 
described is therefore intended to be only illustrative 15 
and not restrictive, since the scope of the invention is 
defined by the appended claims rather than by the 
description preceding them. All changes that fall within 
the metes and bounds of the claims, or equivalents of 
such metes and bounds, are therefore intended to be 20 
embraced by the claims. 

Claims 

1. A multi-functional on-vehicle camera system com- 25 
prising a plurality of cameras (1 1-1 8) mounted on a 
vehicle and a display unit (1 9-21 ) displaying at least 
one camera image, 

characterized in that 

30 

image processing means (47) is provided for 
processing images taken by said plurality of 
cameras, and 

system control means (42) is provided for 
selecting at least one camera image to be proc- 35 
essed in said image processing means (47) 
from the images taken by said plurality of cam- 
eras (11-18) based on vehicle conditions (36- 
41), and for controlling a type of image 
processing performed in said image process- 40 
ing means (47). 

2. The multi-functional on-vehicle camera system in 
accordance with claim 1 , further comprising graphic 
superimposing means (45, 46) for superimposing a 45 
graphic pattern on said image processed by said 
image processing means (47), 

wherein said system control means (42) controls 
both a type and a position of said graphic pattern 
superimposed by said graphic superimposing so 
means (46, 46). 

3. The multi-functional on-vehicle camera system in 
accordance with claim 1 , wherein said plurality of 
cameras include at least two cameras selected 55 
from the group consisting of a rear camera (18) tak- 
ing a rear view seen from the vehicle, right and left 
pillar cameras (16,17) taking right and left rear side 



views seen from the vehicle, right and left door mir- 
ror cameras (1 4, 1 6) taking right and left door mirror 
views seen from the vehicle via right and left door 
mirrors, right and left lateral cameras (1 2, 13) taking 
front side views seen from the vehicle, and a front 
camera (11) taking a front view seen from the vehi- 
cle. 

4. The multi-functional on-vehicle camera system in 
accordance with claim 1 , wherein said display unit 
includes a right monitor (21) displaying an image 
taken by a right door mirror camera (1 5), and a left 
monitor (20) displaying an image taken by a left 
door mirror camera (14). 

5. The multi-functional on-vehicle camera system in 
accordance with claim 1, wherein said image 
processing means (47) produces a plan view image 
based on images of a plurality of cameras (11-18), 
said plan view image including an image of the 
vehicle seen from above. 

6. The multi-functional on-vehicle camera system in 
accordance with claim 1, wherein said image 
processing means (47) produces a panorama 
image based on images of a plurality of cameras. 

7. The multi-functional on-vehicle camera system in 
accordance with claim 1, wherein said image 
processing means (47) produces a joint image by 
combining images of a plurality of cameras. 

8. The multi-functional on-vehicle camera system in 
accordance with claim 1, wherein said image 
processing means (47) produces a surrounding 
image representing a surrounding view seen from 
the vehicle by successively connecting images of 
all of said plurality of cameras mounted on the vehi- 
cle. 

9. The multi-functional on-vehicle camera system in 
accordance with claim 1, wherein said image 
processing means (47) produces an image formed 
by using part of image data obtained from said plu- 
rality of cameras. 

10. The multi-functional on-vehicle camera system in 
accordance with claim 1, wherein said image 
processing means (47) produces an image includ- 
ing a moving object approaching or leaving with 
respect to the vehicle, said moving object being 
emphasizing by a different color. 

11. The multi-functional on-vehicle camera system in 
accordance with claim 1, wherein said system con- 
trol means (42) designates said camera image to 
be used in said image processing and designates 
the type of image processing performed in said 
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image processing means (47) based on at least 
one signal representing vehicle conditions selected 
from the group consisting of a steering angle signal 
(37), a vehicle speed signal (38), gear position 
information (39) and winker information (40). 

12. The multi-functional on-vehicle camera system in 
accordance with claim 2, wherein said graphic 
superimposing means (45, 46) superimposes a 
graphic pattern of a moving object approaching or 
leaving with respect to the vehicle. 

13. An image display method for a multi-functional on- 
vehicle camera system comprising a plurality of 
cameras (1 1 -1 8) mounted on a vehicle and a dis- 
play unit (19-21) displaying at least one camera 
image, said image display method comprising the 
steps of: 

processing at least one camera image selected 
from a plurality of images obtained from said 
plurality of cameras based on vehicle condi- 
tions/and 

displaying a processed camera image on said 
display unit. 

14. The image display method for a multi-functional on- 
vehicle camera system in accordance with claim 

13, wherein 

said plurality of cameras (11-18) include at 
least two cameras selected from the group 
consisting of a rear camera (18) taking a rear 
view seen from the vehicle, right and left pillar 
cameras (16, 17) taking right and left rear side 
views seen from the vehicle, right and left door 
mirror cameras (14, 15) taking right and left 
door mirror views seen from the vehicle via 
right and left door mirrors, right and left lateral 
cameras (12, 13) taking side views seen from 
the vehicle, and a front camera (11) taking a 
front view seen from the vehicle, and 
images of said at least two cameras are proc- 
essed in image processing means (47) and dis- 
played on said display unit (19-21). 

15. The image display method for a multi-functional on- 
vehicle camera system in accordance with claim 

14, wherein 

a plan view image is produced and displayed 
based on at least three images of said rear 
camera (18) and said right and left pillar cam- 
eras (16, 17) when said vehicle moves back- 
ward for parking, said plan view image 
including a view of the vehicle seen from above 
(step 3). 



16. The image display method for a multi-functional on- 
vehicle camera system in accordance with claim 
14, wherein 

5 a plan view image is produced and displayed 

based on at least three images of said rear 
camera (18), said left pillar camera (16) and 
said left door mirror camera (14) when said 
vehicle moves backward for parking along a left 

w side of a road, said plan view image including a 

view of the vehicle seen from above (step 5). 

17. The image display method for a multi-functional on- 
vehicle camera system in accordance with claim 

15 14, wherein 

a plan view image is produced and displayed 
based on at least three images of said rear 
camera (18), said right pillar camera (17) and 
20 said right door mirror camera (15) when said 

vehicle moves backward for parking along a 
right side of a road, said plan view image 
including a view of the vehicle seen from above 
(step 6). 

25 

18. The image display method for a multi-functional on- 
vehicle camera system in accordance with claim 
14, wherein 

30 a panorama image is produced and displayed 

by combining at least two images of said left 
door camera (14) and said left pillar camera 
(16) when said vehicle turns left, thereby elimi- 
nating an accident occurring at an inside of the 

35 vehicle during a left turning or cornering opera- 

tion (step 1 8). 

19. The image display method for a multi-functional on- 
vehicle camera system in accordance with claim 

40 14, wherein 

a panorama image is produced and displayed 
by combining at least two images of said right 
door camera (15) and said right pillar camera 
45 (17) when said vehicle turns right, thereby elim- 

inating an accident occurring at an inside of the 
vehicle during a right turning or cornering oper- 
ation (step 19). 

so 20. The image display method for a multi-functional on- 
vehicle camera system in accordance with claim 
14, wherein 

a panorama image is produced and displayed 
55 by combining at least two images of said left 

pillar camera (16) and said rear camera (18) 
when said vehicle performs lane changing to a 
left lane (step 12). 
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21. The image display method for a multi-functional on- 
vehicle camera system in accordance with claim 
14, wherein 

a panorama image is produced and displayed 
by combining at least two images of said right 
pillar camera (17) and said rear camera (18) 
when said vehicle performs lane changing to a 
right lane (step 13). 

22. The Image display method for a multi-functional on- 
vehicle camera system in accordance with claim 
14, wherein said display unit includes a right moni- 
tor (21 ) displaying an image taken by said right door 
mirror camera (15) and a left monitor (20) display- 
ing an image taken by said left door mirror camera 
(14). 

23. The image display method for a multi-functional on- 
vehicle camera system in accordance with claim 
22, wherein each of said right and left monitors (21 , 
20) displays a limited image seen within a view 
angle of a corresponding door mirror when said 
vehicle is traveling (Fig. 7). 

24. The image display method for a multi-functional on- 
vehicle camera system in accordance with claim 
14, wherein 

a panorama image is produced and displayed 
by combining three images of said right and left 
pillar cameras (1 6, 1 7) and said rear camera 
(18) when said vehicle is traveling ordinarily 
(step 14). 

25. The image display method for a multi-functional on- 
vehicle camera system in accordance with claim 
14, wherein 

a joint image is produced and displayed by 
combining two images of said right and left lat- 
eral cameras (12, 13) when said vehicle is 
traveling slowly or stopped for confirmation of 
forward or side obstacles (Step 15). 

26. The image display method for a multi-functional on- 
vehicle camera system in accordance with claim 
25, wherein 



the image displayed on said display unit 
includes a moving object approaching or leav- 
ing with respect to the vehicle, said object 
being emphasizing by a different color. 

5 

28. The image display method for a multi-functional on- 
vehicle camera system in accordance with any one 
of claims 13 to 26, wherein 

w a graphic pattern is superimposed on the dis- 

played image, said graphic pattern represent- 
ing a moving object approaching or leaving 
with respect to the vehicle. 

15 29. The image display method for a multi-functional on- 
vehicle camera system in accordance with claim 
22, wherein 

the image taken by said door mirror camera 
20 (14, 15) is continuously displayed when any 

passenger is in the vehicle. 
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30 



35 



40 



45 



the image of said right lateral camera (12) is so 
enlarged when said vehicle turns right, and the 
image of said left lateral camera (13) is 
enlarged when said vehicle turns left (Fig. 9). 

27. The image display method for a multi-functional on- 55 
vehicle camera system in accordance with any one 
of claims 13 to 26, wherein 
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